In business applications, time series data can be leveraged for decision support, anomaly detection and monitoring, for example, anomaly detection from IT monitoring data, abnormality identification in trading data, condition-based monitoring in oil platforms, anomaly diagnosis for manufacturing tool, and in other applications. In such contexts, modeling correlation is not enough, and the inventors in the present application have recognized that there is a need to understand and quantify causal effects among the various parameters involved. In addition, in relevant applications the underlying dependency structures may vary over time. Hence, the inventors in the present application have also recognized that there is a need to develop methods that do not assume a static causal graph.
Traditional methods for causal modeling assume a static causal graph modeling. For instance, in traditional methods, a unique static model is learnt and anomaly detection is performed by reference, i.e., by learning a model of “normal” relationships, and performing likelihood evaluation of observed data based on the reference model, or learning another static model for an evaluation dataset and comparing it with the reference model.
No change point detection method exists for temporal causal modeling. Yet in many relevant applications, the underlying dependency structures may vary over time. Anomaly detection provides a good example, where the primary interest is to understand how and when the causal relationships between various variables may have been altered over time. For instance, the existing Dirichlet process (DP)-based methods do not allow state estimation. Certain change point methods only allow for a new state when a change point occurs, e.g., change to a previous state is not allowed. Markov Chain Monte Carlo (MCMC)-based methods are computationally intensive and may be considered impractical.
The detection of and the inference about time series data that may come from multiple regimes is known as the “change point modeling” in statistical literature. However, change point modeling for sparse causal graphical models has not been discussed in the literature. Furthermore, most literature in change point modeling does not allow the process to return to previous states.